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Abstract —Demand-side management (DSM) has emerged as 
an important smart grid feature that allows utility companies to 
maintain desirable grid loads. However, the success of DSM is 
contingent on active customer participation. Indeed, most existing 
DSM studies are based on game-theoretic models that assume 
customers will act rationally and will voluntarily participate 
in DSM. In contrast, in this paper, the impact of customers’ 
subjective behavior on each other’s DSM decisions is explicitly 
accounted for. In particular, a noncooperative game is formulated 
between grid customers in which each customer can decide on 
whether to participate in DSM or not. In this game, customers 
seek to minimize a cost function that reflects their total payment 
for electricity. Unlike classical game-theoretic DSM studies which 
assume that customers are rational in their decision-making, a 
novel approach is proposed, based on the framework of prospect 
theory (PT), to explicitly incorporate the impact of customer 
behavior on DSM decisions. To solve the proposed game under 
both conventional game theory and PT, a new algorithm based 
on fictitious player is proposed using which the game will 
reach an £-mixed Nash equilibrium. Simulation results assess 
the impact of customer behavior on demand-side management. 
In particular, the overall participation level and grid load can 
depend significantly on the rationality level of the players and 
their risk aversion tendency. 

Index Terms —Smart grid, game theory, prospect theory, de¬ 
mand side management. 

I. Introduction 

The use of demand-side management (DSM) mechanisms 
has recently attracted significant attention in the smart grid 
literature. DSM schemes offer smart grid customers an op¬ 
portunity to change their demands over time so as to reduce 
their overall electricity costs. From the utility company’s 
perspective, such a shifting of demand can reduce the peak 
hour demand on the grid m. A successful implementation 
of DSM schemes requires customers to actively subscribe 
to the offers made by the utility company. However, recent 
empirical studies have shown that customers remain reluctant 
to subscribe to DSM mechanisms, despite the efforts of utility 
companies Therefore, it is important to develop a new 
generation of DSM mechanisms that can improve the penetra¬ 
tion of the technology and thus accelerate the deployment of 
the smart grid. 

There has been an abundant body of work dealing with 
DSM iJl-lHl. The authors in O proposed a distributed, game- 
theoretic DSM system, in which each user chooses a daily 
schedule of household appliances and loads to optimize the 
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energy consumption. The work in ||4l proposed different game- 
theoretic approaches to optimize a day-ahead DSM mechanism 
by using storage units. The overall value of implementing 
DSM and demand response schemes was studied in 0 via 
a Stackelberg game formulation. The work in 0 investigated 
how energy consumption may be optimized through a two- 
step centralized model, in which a power supplier provided 
consumers with an energy price parameter and consumption 
summary vector. In Q, the authors adopted an instantaneous 
load billing scheme so as to shift consumers’ peak hour de¬ 
mand and to charge them fairly for their energy consumption. 
In [si, utility companies and residential users were modeled 
in two levels, which reduce demand variation and peak load. 

Thus, the works in 0-0 study the various economic and 
optimization aspects of DSM and are representative of the 
majority of existing works in this area. However, most of 
those existing works assume that customers will act rationally 
and subscribe to DSM as long as their objective function can 
be optimized 0-0. In practice, as observed by numerous 
real-world experimental studies 0-ini, the process of in¬ 
dividuals can deviate significantly from the rational premise 
of conventional game theory. This has been corroborated by 
the relatively low adoption rate of DSM in deployed smart 
grids 0. In this respect, prospect theory (PT), a Nobel- 
prize winning theory developed by Kahneman and Tversky, 
provides the necessary tools to better understand real-world 
decision making and its deviation from rational behavior ifT^ . 
In particular, PT studies have shown that, in real life, users 
often have subjective behavior when faced with uncertainty of 
outcome, such as lottery outcomes. Also, it has been shown 
that customers may have subjective perceptions on how others 
behave and how this behavior impacts their gains or losses in 
economic-oriented scenarios. Since the price in DSM strongly 
depends on aggregate demands and thus on the participation 
of all customers, the overall perceptions of the customers on 
each other’s DSM decisions will impact their behavior. PT has 
been widely used in the social sciences ns. In addition some 
recent works m-Ei have applied PT to wireless networks. 
However, to the best of our knowledge, beyond our early 
work in M that studies a two-player storage management 
PT game, no existing work seems to have investigated how 
PT considerations impact participation in DSM. 

The main contribution of this paper is to propose a new 
framework for DSM that accounts for realistic customer partic¬ 
ipation behavior using prospect theory. In particular, the DSM 
problem is cast as a noncooperative game between customers. 
In this game, each customer can decide whether or not to 
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participate in a DSM program, while the utility company seeks 
to reduce the total peak hour demand so as to maintain a 
desirable target load on the grid. The proposed game uses PT 
to explicitly incorporate the customers’ subjective perceptions 
on DSM decisions and their impact on potential cost savings. 
Here, subjective perceptions pertain to the way in which 
each customer evaluates its electricity payment and how that 
depends on other customers’ actions. In this respect, each 
customer is seeking to minimize a cost function that reflects 
its one-day electricity bill, under other customers’ participation 
and its impact on the price. 

Compared to related works on DSM Ei-ii, ua-iiia, 
this paper brings forward novel ideas from PT in order to 
explicitly account for realistic customer behavior, which can 
differ significantly from the classical, rational path predicted 
by traditional game theory. To solve the game, under both 
PT and classical game theory, we propose a new learning 
algorithm that allows the customers to interact and reach an 
e-mixed Nash equilibrium. We prove the convergence of the 
algorithm and discuss the properties of the reached operating 
point. Extensive simulation results based on realistic data show 
that deviations from rational behavior can strongly impact the 
overall level of participation in DSM. 

The remainder of the paper is organized as follows; Sec¬ 
tion |II] presents the studied system model. In Section [Till we 
formulate the problem as a PT-based game. In Section |IV] 
we introduce the concept of fictitious play and describe 
our proposed algorithm. Simulation results are presented in 
Section |V] while conclusions are drawn in Section |V|1 


II. System Model 

A. Demand cost 

Consider a smart grid consisting of a set J\f of customers 
in which each customer i G Af consumes a certain amount 
of energy per hour. All customers are offered the opportunity 
to participate in a demand side management scheme provided 
by the utility company. For customer i, we define an hourly 
energy demand scheduling vector in line with existing works 
such as, 

x, = [xl,x‘^,...,xf], (1) 


where H = 24 hours. For a certain hour h G H = 
{1,2,..., H}, x{j represents the energy demand of customer 
i. The total energy demand from all customers at time h is 
thus 

d^ = Y.xl ( 2 ) 

i&JG 

At a given time h, we assume that the price per energy unit 
charged to a customer i is dependent on its fraction of the 
total current load as follows CD: 


c'lix) = B 




( 3 ) 


SAA • 


where B could be designed based on notions from a locational 
marginal pricing (FMP) scheme ||25]| in which the price 
function is not necessarily time-dependent. The electricity 
price for each customer as given by Q allows one to allocate 
the price for the amount of energy that a customer consumes. 


For example, as the number of customers increases, the total 
demand will increase. Then, the electricity pricing must reflect 
this change in the demand. For each customer, the individual 
electricity pricing is dependent on its demand proportion as 
captured by Q. Hence, the total cost of user i over H hours 
is given by 

H 

( 4 ) 

h=l 

Given the price and local demand, an energy market is set 
up in which all customers seek to minimize their costs while 
maintaining their energy demands at a desired level. Here, 
all customers will interact so as to determine their demands 
under DSM. These demands include the quantities of energy 
required and impact the price at a certain time. Instead of 
fixed reservation prices announced by a utility company, in 
our model, each user can strategically change its demand, in 
which its demand fraction of the total demand impacts the 
underlying electricity price. In this respect, the price in ([3ll can 
lead to an increased cost for all customers, including those that 
decide not to participate in load management. Moreover, the 
demand delayed by DSM will cause a varying electricity price 
in subsequent hours. Thus, we will develop a DSM mechanism 
based on load shifting. 


B. Load Management 

In this subsection, we propose a load shifting mechanism 
to analyze the variations of the demand over time. Inherently, 
load shifting allows part of the peak hour load to be moved to 
an off-peak hour, in which such controlled demand is delayed 
to the following time slot in our formulation. For the proposed 
model, we assume that, for each participating customer, the 
demand can be adjusted so as to meet a predefined/target 
energy demand profile (vector) set by the utility company, 
given by 

Gd = [g\,gl,---,gd]- ( 5 ) 

On the one hand, a utility company wants to reduce peak 
hour demand in order to decrease load on the grid. On the 
other hand, the company wants to maintain the amount of load 
shifted within a reasonable range while avoiding the creation 
of a new peak hour. For example, if the power company wants 
to reduce 10% of the load at a given hour H, the target energy 
demand could be defined by 



if h = H, 


( 6 ) 


where /3 = 0.9 and gh G [g^,g‘^, ■ ■ ■ ,g^] can be the historical 
demand referenced by the utility company. Here, we assume 
that a customer can determine when it starts to participate in 
DSM. In this respect, customer i will choose its starting time 
(defined as the participating time at) so as to minimize its cost 
by observing the difference between the predefined demand 
per user and its daily demand. In particular, we assume that a 
customer i would not leave DSM after choosing to participate 
over H = 24 hours and thus, the number of participating 
customers at a given time h is = \I^\ = \{ai > h}\. 

Here, is the starting hour and we discuss its impact in 
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more detail in Section IIII-AI In this case, the reduced demand 
of participating customer i is given by 


.h 

I 



9d XligAryz ^ + 
jn ) 1 


if 

if ffS > 


(7) 


where (< 7 )+ := max( 0 ,( 7 ) and is the total demand that 
includes both and the amount shifted from the previous 
hour h — 1, such that 0 < 7 i < 1 is a 

factor by which customer i wants to reduce from exceeding its 

demand, as — - ‘ is the averaged demand suggested 

by the utility company for all participating customers. In 
particular, if the demand of participating customer i is less 
than the average demand for participating customers, i.e., 

, its reduced demand is rf = 7 ^ • 0 = 0 . 

Using (|7]), if customer i participates in DSM at time h, its 
demand will be 

y^ = x’l-Tl ( 8 ) 


Then, the participating customer i moves its shifted load to the 
following hour, and thus, its DSM demand at time h < t < H 
is 

yl = (xl + 4 -^) - rl (9) 

while the demand at the final hour H is 


2/f = xf+ 


( 10 ) 


To analyze such a load management scheme, we next 
propose a new framework that builds on the analytical tools 
of classical game theory and prospect theory. 


III. Game-theoretic Formulation for Demand-Side 

MANAGEMENT 

In this section, we first formulate a noncooperative game 
between the customers, and then study the proposed load 
shifting model using expected utility theory 12 ^ and prospect 
theory ifT^ . also discussing their various properties. 

A. Noncooperative Game Model 

In order to analyze the interactions between customers, we 
use noncooperative game theory ll26ll . as the strategy choices 
of the customers are interdependent. We formulate a strategic 
noncooperative DSM game S = (Af, {A}igaA, 
where J\f is the set of players, the action € Ai := 
{ 1 , 2 ,. ..,iF} of customer i is the time (hour) at which cus¬ 
tomer i would like to begin participation in DSM, and the cost 
function Ui of customer i captures its electricity payment to the 
company, using the price in Q. Here, we note that, although 
a customer can participate in load management, its total daily 
demand remains the same (i.e., = Ef=i y|)- Thus, 

the utility function (cost) for a player i G Af that chooses an 
action Oi is given by 

CL~i) — 'y ' Cj f ^ y^ (fli, CL^i) j X iflii 
h—1 

( 11 ) 

where a_i = [ 01 , 02 ,... ,Oi_i,ai+i,... , 0 ^]'^ is the vector 
of action choices of all players other than i, and j/f (o^, O-i) is 


the DSM demand of user i, compared to the initial demand 
in O. For example, each customer can determine a starting 
hour Qi and shift its load to after that hour. 

The goal of each customer i is to choose a strategy o^ G Ai 
so as to minimize its cost as given in dm i. For characterizing 
a desirable outcome for the studied game S, one must derive 
a common solution that enables one to capture the coupling 
between the individual optimization of each customer. We 
define a = (oi, a_,;) as the vector of all players’ strategies. For 
each such vector a, we will have a different electricity price 
c[*( ) in (|3ll. Prior to finding a solution for the proposed DSM 
game, we will introduce expected utility theory and prospect 
theory. 

B. Expected Utility Theory (BUT) 

In a smart grid, as the customers may, over a long time 
period, change their DSM preferences, we are interested in 
studying the frequency with which they choose a certain 
time to begin DSM participation. Therefore, we mainly study 
the proposed game under mixed strategies ll26l so as to 
capture the customers long-term, probabilistic choices of a 
DSM start time. Let p = [pi,...,ppf] be the vector of 
all mixed strategies, where, for every customer i G Af, we 
have Pj = [pi{l),... ,pi{H)]'^ and Pi{ai) is the probability 
corresponding to the choice of a pure strategy Ui G Ai- 

Under the conventional EUT model, the expected cost of 
customer i is captured via the expected value over its mixed 
strategy. Computing each user’s utility requires the vector of 
all players’ strategies. In particular, we assume that the smart 
grid communication infrastructure will make this information 
available to users who participate in DSM. Thus, the EUT cost 
of a player i will be given by 

Uf^^ip)= X] (12) 

aeA ^ 1=1 ' 

where a is the vector of all players’ strategies and A = 
Ai X A 2 X ••• X An- In particular, the mixed-strategy 
p{h) = [pi{h),p_^{hy\^ represents the empirical frequencies 
with which the customers choose a certain participation time 
h. These frequencies capture how often customer i participates 
in DSM. p_^(h) is the vector of mixed strategies of all players 
other than i, corresponding to a_i in dm i. 

C. Prospect Theory 

As previously mentioned, a player can evaluate its expected 
utility by using ([T 2 li . in which case the customers are assumed 
to act rationally and objectively under EUT. However, in real 
life, individuals do not truly behave rationally nor do they trust 
the rationality of others’ behavior. Thus, in order to develop a 
realistic model of DSM, one must account for the fact that, in 
practice, customers may not assess their utilities objectively. 
Indeed, it has been shown that, despite the benefits of DSM, 
its adoption in practice has remained slow, due to unexpected 
customer behavior m. 

To study such realistic/practical participation models, we 
will develop a DSM game model, in which a customer may 
have subjective views on how the opposing players will choose 
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their strategies. This, in turn, impacts the way in which this 
customer evaluates its utility in (fTTl i which depends on other 
customers’ strategies. Indeed, it has been observed that in 
real-life decision-making, people tend to subjectively weight 
uncertain outcomes El. Uncertainty, here, is defined as the 
fact that the amount of utility that a customer will obtain, 
depends on the decision making behavior of other customers, 
which is probabilistic. For example, a customer cannot be 
sure of the time at which other customers will begin their 
participation in DSM. Thus, the customer will not be sure 
how much economic gain its participation in DSM will yield; 
since such a participation depends on all the players’ strategies. 
Thus, this customer’s EUT evaluation as per (fTSli might be 
overweighted or underweighted due to the subjective observa¬ 
tion of others. Moreover, this customer may need to properly 
assess whether to shift its demand or not, as it is unsure of 
other customers’ strategies. In this respect, a customer might 
have its own subjective perception on the participation of other 
customers in the DSM game (and on the actual price), thus 
deviating from the rational assumption of conventional game 
theory and EUT. 

In order to analyze such subjective perceptions, we will 
use the behavioral framework of prospect theory Ha. In this 
studied model, we mainly focus on how a customer evaluates 
the strategies of its opponents and, thus, acts accordingly. Our 
focus is on the prospect-theoretic perspective, which primarily 
deals with how human decision makers deal with economic 
outcomes that have some form of uncertainty. 

One important PT concept is the so-called weighting effect. 
'Weighting effect naturally implies that customers will have 
subjective views on how their opponents will act (i.e., a 
weighted view on the probability vector of those players), 
which, in turn, maps to subjective views on their expected 
utilities. In the proposed game, we use this weighting effect to 
incorporate a subjective evaluation for each user’s observation 
of the mixed strategy of its opponents. Thus, under PT, instead 
of objectively observing the mixed strategy vector chosen 
by the other players, each user perceives a weighted version 
of it, Wi{p—i). Wi{-) is a nonlinear transformation that maps 
an objective probability to a subjective one. PT studies have 
shown that most people will often overweight low probability 
outcomes and underweight high probability outcomes 02. 
Without loss of generality, we assume that all players use a 
similar weighting approach, such that tUi(-) = w{-), Vi G Af. 
Although many weighting functions exist, we choose the 
widely used Prelec function (for a given probability cr) El, 

tt;(CT) = exp(—(—lnCT)“), 0 < a < 1, (13) 

where a is a parameter used to characterize the distortion 
between subjective and objective probability. Note that when 
a = 1, the weighting effect will coincide with the conventional 
EUT probability. Eig. [T] illustrates the probability weighting 
effect. In this figure, we can see that the objective probability 
and subjective estimation insect at p = 1/e and the curve 
approximates EUT as a increases. Also, weighted summations 
are not necessarily equal to 1 (i.e., r(;(0.4) -f w{0.6) < 1). 

Under PT, the expected utility achieved by a player i, given 
the weighting effect, is 



Fig. 1. The subjective probability under the weighting effect, as the objective 
probability varies. 

UriP)= ^(pi(a*)]^u;(pi(ai)) (14) 

Here, we assume that a player has a subjective evaluation 
only of the other players’ strategy probabilities. In this respect, 
customer i’s subjective evaluation of its own probability is 
equal to its objective probability. 

Having defined the cost functions under both EUT and PT, 
our next goal is to find a solution for the game. Given the set 
of probability distributions Vi over its set of strategies Ai, a 
suitable solution of the game would be a mixed-strategy Nash 
equilibrium defined as follows: 

Definition 1: A mixed strategy profile p* gV = Oili 
is a mixed strategy Nash equilibrium (NE) if, for each cus¬ 
tomer i G {1,2, ...,V}, we have {Ui represents and 

up- under EUT and PT, respectively) 

Ui{p*,P~i) < U^(Pi,P-i), ^Pi e (15) 

In practice, to avoid slow convergence times and unnecessary 
overhead, we can consider approximate equilibrium solutions 
that allow the players to reach the neighborhood of an equi¬ 
librium. Hence, we mainly focus on e-Nash equilibria which 
are defined as follows: 

Definition 2: A mixed strategy profile p* G V = 
is an e-Nash equilibrium (e-NE) if, for every player i, we have 

U^{.plpU) < U,{p,,pP) + e, Vp, G V,, (16) 

where e is a small positive number. 

In order to find the solution for the game 5 under both 
EUT and PT, we find a mixed e-NE in strategic form which 
represents a point within a close neighborhood of the exact 
equilibrium ll26l . 


IV. Game Solution and Algorithm 

In this section, we propose a novel algorithm, under both 
EUT and PT, to solve the studied DSM game and find an 
equilibrium point. The proposed algorithm builds on classical 
fictitious play (EP) ll28ll . Thus, we propose the following iter¬ 
ative algorithm to find an e-Nash equilibrium of the proposed 
game: 

' -{v^-pP), (17) 


(fc+i) (fc) 
Pi -Pi 


A 


k-\- 1 
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where 0 < A < 1 is an inertia weight, k is the iteration index. 


and (ai),( 02 ),..., t;-"''is a vector of 

player i’s strategies such that, v^j^\ai) = 1 if player i chooses 
the hh strategy at iteration k and v'^\ai) = 0 for updating 
the strategies excluding the kh strategy. The pure strategy i.e., 
the kh strategy, is the one that minimizes the expected utility 

with respect to the updated empirical frequencies. Thus, player 

ih) 

i can repeatedly choose its strategy and update v\ ’ as follows: 




,(fc) 


(fe)/ 


(fc)/ (fe)N 

v\ ') = 


'l,if =arg min 

ai^Ai 


(18) 

^0, otherwise, 

where the utility here is the expected value obtained by 
player i with respect to the mixed strategy of the opponent, 
when player i chooses pure strategy a; for BUT and its 
weighted mixed strategy w{pi{ai)) for PT. It is well known 
that our algorithm (as a simplified iterative approach of smooth 
fictitious play (SFP)) is guaranteed to converge to a mixed e- 
Nash equilibrium ll29l . as the players’ beliefs (probabilistic 
choices) converge to a fixed point. In particular, a player’s 
belief is its mixed strategy set, and e represents the utility 
difference based on beliefs. Here, we define Cp as the mixed 
strategy difference corresponding to the utility. SFP can reach 
an e-NB under BUT, in which the belief difference Ep between 
two iterations is small when the number of iterations k goes 
to infinity. However, to our knowledge, such a result has not 
been extended to PT. Here, we first prove that the proposed 
algorithm in (fTTl i will converge to a fixed point in beliefs, as 
follows: 

Theorem 1: There exists an inertia weight A, 0 < A < 1, 
such that the iterative algorithm in (fTTl i converges to a fixed 
point in belief, as a mixed ep-equilibrium (Cp < -^^)- 

Proof: For the proposed DSM game, the proposed SFP 
process is guaranteed to converge to a fixed point in be¬ 
liefs 1^ under BUT. Here, we mainly prove that, under PT, 
a player i’s iterative sequence {Pi(fc)} converges to a mixed 
ep-equilibrium in beliefs. From (fTTl i. we have 


Pi 


(fe-ri) 


1 - 

A ^ 

\ (fc) 

jPi 


A 

^(fc) 

A: + 1 y 


fc -f 1 


1 - 

1 ' 

\ (fc) 

jPi 

+ 

1 

(fc) 

vl 

k 1. 

fc -f 1 


k + V^ 


(19) 


1 - A 


,(fc) 


k + l ‘ ’ 

where the first two terms represent the iteration using FP 
and, we define 

1 - A 


1 - A (fc) 
- Pi 


,{k) 


( 20 ) 


^ k + V^ k + 1 ^ 

In (dg and (fTOll, we present the belief difference between 
the proposed algorithm and FP, where there exists an Cp 
difference at the fcth iteration. In particular, the distance 
boundary between FP belief and the approached belief can 
be bounded as follows: 


1 - A, 
k + 1 


kp.™ - »r')i < 




1 - A 
k + 1 


■V2, 


( 21 ) 


where gp becomes small as k increases. Indeed, the Buclidean 
distance in (fTTl i involves in (fTSl l. one of whose compo- 

(k) 

nents is 1, and the mixed strategy set p: , whose components 


will sum to 1. Thus, the maximum Buclidean distance between 
and is less than at the A:th iteration. 

I fc+1 

Within the given boundary, (US can be rewritten as 


p‘‘+‘'=[i- 


A 


k + 1 
A 


P 


(fc) 


k + \ 


,(fc) 


fc -I- 1 


1 ^ 1 


(fc-i) 


,(fe) 


fc -I- 1 


1 - 


A A A 


(fc-i) 


= 1 - 


k + 1 1 k 


1 - 


fc -f 1 


-f 


A 


+ ( 1 - 


fc-fl * 

k 


1 2 


A A A (h- 


( 1 ) 


k+l k * 


(fc-i) 


A xAA 


nwjTi) 

i=2 ■' 


.(1) 


2^^* ■ 


( 22 ) 


The first term in (l22li is ~ Since 

is decreasing as fc increases and bounded 

by \pf\ < 1 , the first term is convergent as fc goes to infinity. 
In fact, this term converges to 0. 

For the remaining terms in (l22l i. we can define 




A 

j + 1 


, 0 ) 


(23) 


Similarly, since \hj\ is decreasing as fc increases and bounded 
by = 1, 1® convergent as fc goes to infinity. 

Thus, we conclude that, will converge to a fixed point, 

as a mixed ep-equilibrium in beliefs using (fTTl i. ■ 

Remark 1: When A = 1, the proposed algorithm in (fTg is 
reduced to FP. Hence ( fTg and (l22l i can be derived as = 

H-+ Thus, FP (A = 1) might 

cycle if repeats after some iteration, i.e., = 

= • • •, and = • • •, while the proposed 

algorithm (0 < A < 1) in (fTTl i converges to a fixed point. 

Theorem 2: For the proposed DSM game, the proposed 
algorithm in (fTTl i is guaranteed to converge to a mixed e-NE 
under both EUT and PT, as its beliefs converge to a mixed 
ep-equilibrium. 

The proof of Theorem |2] is given in the appendix. 

We summarize the proposed DSM solution in Table U To 
find the solution of this game under both EUT and PT, we 
use the algorithm in (fTTl i to solve for the mixed e-NE. In the 
first phase of the algorithm, each customer will set an initial 
probability vector, while any non-participating customers will 
set its probability to 0 throughout the whole DSM process. At 
the beginning, each participating customer observes others’ 
strategies and evaluates its expected costs of every strategy. 
In the evaluation process, a participating customer can receive 
an estimate of the expected costs from the grid operator under 




























6 


TABLE I 

Proposed Load Shifting Solution 


Phase 1 - Proposed Dynamics: 

Each customer i G Af chooses a starting strategy as its mixed 
strategy of participation. 

repeat, 

1) Each customer i G Af observes others’ participation p_^ 

2) Each customer iGAf updates its better response strategy using 

the proposed algorithm in (IIJ: 

i) The utility operator communicates with the customer 
using the grid’s two-way architecture communication 
(see (H or ^ and references therein). 

ii) Customers’ loads can be shifted via DSM as in Section Hl-BI 
Load Shedding 

a) The utility operator advertises all customers’ participation 
times ai G Ai using their mixed strategies. 

b) Each customer publishes its participation based on a 

mixed strategy representing the subjective observation 

with underlying weight effect. 

c) After combining probabilities as in o, customer i will 
receive an expected cost under EUT, sent by the operator, 
and observe the curi'ent vector of strategies p_i so as to 
assess its subjective utility in (H under PT. 

d) Customer i chooses its strategic response in GD- 
until convergence to a mixed NE strategy vector p*. 

Phase 2 - Power Company Strategy 

1) The operator receives the participation information 

given the mixed strategy set as per a*. 

2) Actual load shifting is implemented under realistic participation. 

EUT, since the grid operator objectively collects information 
and provides required services/information. Then, the cus¬ 
tomers will overweight or underweight the information about 
others’ strategies based on the individual weighting effect in 
O- Moreover, each customer will subjectively estimate its 
expected costs of every strategy and then frequently report its 
participation based on its mixed strategy. The communication 
between customers and grid operator will continue until the 
expected costs satisfy the setting e in utility (fT^ . (fT4l) and (fThl) . 
Once a mixed e-NE is reached, the customers will frequently 
signal their participation decisions based on probabilities and 
participate in demand-side management in Phase 2. This phase 
of the proposed load shifting solution is the practical market 
operation. Given the submitted information, the grid operator 
will shift/reduce local loads. The actual process of Phase 2 
is beyond the scope of this paper and will follow real-world 
contract negotiations that could be interesting to study in future 
work. 

V. Simulation Results and Analysis 

Eor our simulations, we use the real-world load profile 
in on which represents customers’ initial demands, i.e., the 
data between April 29th, 2013, and May 4th, 2013, from 
Miami International Airport, since the data of local customer 
houses, or groups, is large, private and confidential. In all 
simulations, each customer can choose a starting time to 
participate in DSM from the time period between 18:00 and 
20:00. Alternatively, the customer can decide not to partic¬ 
ipate; that is, Ai = {18,19, 20, 24}, Vi G Af. Also, we set 
(3 = 0.86 and 7 = 0.6 for all customers. 

Pig. |2] shows, for a smart grid with six customers, all 
mixed strategies of a selected Customer 1 under both EUT 
and PT. In this figure, we choose a = 0.7 for all customers 
to represent their distortion under the weighting effect in 


Customer 1’s mixed strategy e-NE 



Fig. 2. The probability performance of all mixed strategic participations for 
Customer 1 under both EUT and PT. 


All Customers’ mixed strategy e-NE 



Fig. 3. The mixed strategic participations for the six customers using both 
EUT and PT at 19:00. 

O- Clearly, Customer I’s PT strategy is different than its 
EUT strategy. Compared to the EUT results (solid lines), 
the PT participating strategy (dash line) at 20:00 is larger, 
while the other PT mixed strategies are smaller. In essence, 
we observe that PT generally enhances (reduces) high (low) 
frequency EUT strategies. Eor instance, under EUT, Customer 
1 will have the highest participation strategy at 20:00 due 
to load shifting as per Q and the varying price in Q. 
In particular, its largest mixed strategy is greater than the 
average mixed strategy, i.e., 0.25 in the proposed four-strategy 
game. Second, because of the weighting effect in (13), in 
Fig.m we can see how PT behavior can be different from 
EUT. In particular, we observe that a PT customer wants to 
participate more than in the EUT case at times when it has a 
low hourly energy prices in Q, which also corresponds to a 
situation with high participation under EUT. In other words, 
for minimizing its cost, under PT, each customer tends to 
increase its participation at the hours with lower hourly prices 
in (O and to decrease its participation at the hours with higher 
hourly prices. This is mainly due to the relationship between 
shifting the demand and the resulting price as captured by 
(|3ll and (fTTl i. Thus, under realistic behavioral considerations, 
customers will accentuate the traditional behavior perceived 
under EUT. This will consequently impact the overall DSM 
performance as shown in the next figures. Accordingly, under 
PT, we can observe that higher EUT probabilities will become 
more pronounced. Thus, the largest mixed strategy using EUT, 
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Fig. 4. The expected nonparticipating load for the six customer game under 
mixed strategies using both EUT and PT over 24 hours, when all customers 
have the same a = 0.7. 
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Fig. 5. The expected nonparticipating load for the six customer game under 
mixed strategies using both EUT and PT over 24 hours, when customers have 
different values of a. 


i.e., at 20:00, would be overweighted via PT observation, and 
vice versa. This is due to the fact that each PT customer takes 
a risk by participating when the hourly costs are low. 

Fig. m shows, using the same parameters as Fig. |2] the 
probability that each customer participates in DSM at 19:00 
under both EUT and PT. In this figure, we can first see that the 
mixed strategy of each customer using PT is different from that 
resulting from EUT. Under EUT, the rational mixed strategies 
made by some customers, such as Customers 3-6, are higher 
than the average mixed strategy, because they have low costs at 
19:00. In particular, given the price in Q, a customer’s lowest 
demand between 18:00 and 20:00 would cause the lowest 
costs and the highest participation. This implies that a rational 
customer wants to participate in DSM when it does not require 
a lot of energy (or its demand is low) in practice, since the 
payment is low. Under PT, the customers’ realistic decisions 
will impact the price in © and change their participation 
levels. In Pig. [3] some customers’ mixed strategic components 
using PT are greater than those of EUT, i.e.. Customers 3-6, 
while Customers 1 and 2 have a lower participation probability 
using PT. If four PT customers, based on their loads between 
18:00 and 24:00, decide to shift more load at 19:00, their 
PT demands will be less than EUT. Then, compared to EUT, 
the PT prices of Customers 1 and 2 will increase because of 
the changing load fraction in ©. Because of the increasing 
prices. Customers 1 and 2’s costs increase and they will be 
less interested in participating at 19:00. Indeed, there will exist 
some customers, such as Customers 1 and 2, who want to 
decrease their participation at 19:00 under PT as shown in 
Pig. 121 Then, the overall demand increases and Customers 3-6 
obtain a decreasing pricing in © which will make these four 
customers more likely to participate at 19:00. 

Pig. m shows the expected nonparticipating load profile 
in the proposed DSM game as time varies. We choose the 
same customers as in Pig. |2] with the distortion parameters 
are set to a = 0.7 for all customers. The actual demand 
is the summation of initial demands in ©. In Pig. 01 the 
nonparticipating load is the minimum expected load that the 
power company must supply, while the participating load 
represents the load that can be partly shifted based on each 
individual customer’s action in ©. Here, all customers can 


start the game from 18:00 and the expected nonparticipating 
loads between EUT and PT are different. On the one hand, the 
difference between EUT and PT during 18:00 and 20:00 is due 
to the change in the customers’ decisions, as previously shown 
in Pig. [ 3 ] On the other hand, between 21:00 and 23:00, the 
customers nonparticipating load using PT is always less than 
that using EUT. Indeed, if we translate the proposed game into 
a “participate or not participate” game, 21:00 can be used to 
distinguish two such strategies, and customer behaviors before 
21:00 can impact their participation at a later time. However, 
the fact that the nonparticipation level for PT is less than that 
for EUT after 21:00 directly relates to the choice of a distortion 
parameter a. In particular, a small deviation from the rational 
strategy for a = 0.7 leads to an increased competition between 
the customers due to the fact that a weighted observation 
increases the costs in (El)- Such small deviation represents 
a case in which a customer participates in DSM but does not 
trust its view of the opponents’ strategy or the information 
received from the power company. Thus, a slight deviation 
from the rational path causes increasing costs and customers 
are more apt to shift their loads under PT to decrease the 
impact of being non-rational, compared to EUT. As a result, 
after 21:00, as seen in Eig. 01 the PT nonparticipating load 
will be less than that for EUT. At 24:00, the power company 
needs to deal with the remaining load as defined in ( fTOb . 

Compared to Eig. 0] Eig. |5] shows the expected nonpartic¬ 
ipating load profile using different distortion parameters, as 
time varies. The distortion parameter a in ( fT3] ) allow us to 
measure how a customer perceives the actions of its opponents. 
A large a implies a small distortion, while a small a represents 
an excessively subjective perception. In particular, we choose 
a = [0.5 0.5 0.2 0.1 0.1 0.1]^. In this figure, we can see that, 
when some customers have a very irrational observation of 
their opponents, the PT nonparticipating load between 21:00 
and 23:00 will be higher than EUT. This implies that, in 
reality, if some customers deviate significantly from their 
rational strategies (for example, a customer forgets to assist 
the power company in load shifting), the power company 
will not be able to shift the total load predicted by the 
rational, objective model. Thus, the power company can use 
the distortion parameter to decide on how to design its DSM 


































































The nonparticipating load profiie at 19:00 



varies. 

scheme. 

Fig. | 6 ] shows the expected nonparticipating load at 19;00, 
as the weighting effect parameter a varies. In this figure, we 
can see that the expected nonparticipating load under EUT is 
65.7% of the total load and, the nonparticipating load under 
PT is less than EUT when ol > 0.56. This is because the 
majority of PT customers have more interest in participating 
at 19:00, as shown in Pig. [3 Thus, the power company can 
shift more load in practice, compared to EUT. Also, this figure 
shows that there exists a distortion threshold, such that, if a 
is greater (smaller) than the threshold, PT customers will have 
lower (higher) nonparticipating loads than EUT cases. A large 
distortion parameter, or a small deviation from EUT, yields an 
increased competition thus raising the costs to the customers. 
Consequently, the customers will become risk seeking and 
more apt to shift their loads and decrease their costs. Thus, the 
increasing PT costs will force the majority to shift more loads, 
compared to EUT. However, a small distortion parameter, or 
a large deviation from EUT, will lead to highly distorting 
behavior from the customers which will lead to increasingly 
high competition and decreasing participation, as customers 
become extremely risk averse and unwilling to participate 
in the DSM process. In a nutshell. Pig. | 6 ] shows that a 
small deviation from EUT may be beneficial for the power 
company as it increases customers’ participation. In contrast, 
a significant deviation from EUT will inevitably lead to highly 
risk averse behavior which will prevent most customers from 
participating; thus yielding detrimental results for the grid and 
preventing the operator from reaping the benefits of DSM. 

In Pig. |7] we show all PT mixed strategies of Customer 1 
(corresponding to Pig.[2]i, as the number of iterations increases. 
The proposed algorithm can be shown to converge suhlinearly 
by stud^in^ its rate of convergence which is defined as R = 
1™ ^ ■ Here, we mainly test the convergence of the 

proposed algorithm in (fl7b . In particular, Customer I’s initial 
probability is = [0.2500 0.3333 0.2083 0.2083]'^ and it 
converges to p* = [0.1110 0.1480 0.6484 0.0925]"^. Prom 
Pig. 121 due to the low hourly costs. Customer 1 will increase 
its participation at 20:00, corresponding to the highest EUT 
frequency. In Pig. |7] since the initial probability at 20:00 is 
small, i.e., p^i\a 3 ) = 0.2083, Customer 1 will choose its best 
strategy and update vi (03) = 1 in (fTSl) . Thus, we can see that 


Customer I’s mixed strategy e-NE 



Fig. 7. The probability performance of mixed strategy for six customers based 
on EUT and PT. 


TABLE II 

Customer I’s All Mixed e-NE under both EUT and PT 



EUT 

PT 

1 

[0.1667 0.2222 0.4722 0.1389]-'' 

[ 0.1110 0.1480 0.6484 0.0925]'' 

2 

[0.2667 0.1333 0.5333 0.0667i' 

io.l776 0.0888 0.6891 0.0444]'' 

3 

[0.0392 0.1569 0.4509 0.353oi''' 

io.0261 0.1045 0.6343 0.235li''' 

4 

[ 0.1159 0.2609 0.5072 0.1159i-'' 

io.0772 0.1738 0.6718 0.0772]'' 

5 

[0.2222 0.0889 0.4666 0.2222]'' 

[0.1480 0.0592 0.6447 0.1480]'' 


the participating probability at 20:00 increases, as the iteration 
k increases. Similarly, because of the large initial probabili¬ 
ties at other strategies. Customer 1 will decrease underlying 
participation via updating t;i(ai) = ^ 1 ( 02 ) = '^ 1 ( 04 ) = 0 in 
dH. Also, Table H shows Customer I’s all mixed e-Nash 
equilibria under both EUT and PT. Using different initial 
probability vectors the proposed algorithm in (flTl i will 
reach different Nash equilibria, and the differences between 
EUT and PT at two different Nash equilibria are not the same. 

VI. Conclusions 

In this paper, we have introduced a novel approach for 
studying the problem of demand-side management. It is of 
interest to factor in explicitly the behavior of customers in 
DSM. In particular, we have developed a game-theoretic 
approach, based on prospect theory, using which each player 
subjectively observes other players’ actions and determines its 
own actions so as to minimize a cost function that captures 
the electricity cost over 24 hours. Then, we have proposed 
an algorithm and have proved that it reaches a mixed e-NE. 
Simulation results have shown that deviations from classi¬ 
cal, objective game-theoretic DSM mechanisms can lead to 
unexpected results and loads on the grid, depending on the 
level of the customers’ subjective perceptions of each others’ 
actions. Therefore, whether a customer participates in DSM 
or not, depends on this level of rationality of the customer. In 
a nutshell, the results of this paper have provided important 
insights into the factors underlying the modest participation in 
DSM schemes observed in real-world smart grid systems. 

Appendix 

A. Proof of Theorem \2} 

Proof: The convergence of the beliefs to a mixed Cp- 
equilibrium is shown in Theorem [T] Based on Theorem [H the 
convergence of a mixed e-NE under EUT is a known result 
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for SFP in Il29l . The following proof mainly focuses on PT 
and we prove this theorem by contradiction. 

Suppose that {p^,} is an iterative process resulting from 
the proposed algorithm that converges to a mixed strategy p* 
(an Cp equilibrium) after k iterations. Also, assume a mixed 
NE 6 * = {(5*, Jlj} is near to the fixed point p*. Based on 
contradiction, if the point p* = {p*,p’Li} is not an e-mixed 
NE, there must exists a strategy G p*, such that 1) 

Pi{ai) > 0,pi{ai) € p* (at least one mixed strategy of player 
i is not zero), and 2) 


PT / r „* 


Uz ai,p_, >u, a^,p_i 


PT / s* 
u, ai,d_ 


+ ti at, >Ui a,, 5*_i + eJ a,-, 5 


'^11 ^ —I I 5 


where Ui(ai,p*_j) is the expected utility of pure strategy Ui 
and ei{ai,6*_^) is the utility difference between NE and e- 
NE. Here, since the iterative mixed strategy decreases as the 
number of iterations n increases (n < k), the utility distance 
Eg of a pure strategy between two neighboring iterations/steps 
must be less than a value after a certain iteration k. In 
particular, if the proposed algorithm converge to a fixed Ep 
equilibrium at iteration k and it is not an e-NE in belief 
(utility), we can find an Eg, such that 


0<e,<2 




0 < 2es < u^{a^, d*_^) + Ei{a^, 6%) 




where ^ 

the proposed algorithm process satisfies 


a^A 

> a*_i)w^ip*_i)] 

= Ui{ai, a*_i)w,{S%)\ 

^aeA ' 

cl^A 


+ Ei{ai,5*_i) - Eg 
+ ^-i) + Cs 


Thus, player i would not play but a' after the nth iteration, 
and we will have Pi{ai) = 0 and W-i{pi{ai)) = 0 (the 
other player’s observation on pi). Here, we have proved that, 
Pi{cLi) = 0 contradicts the initial assumption Pi{ai) > 0; thus 
the theorem follows. ■ 
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